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power  increases  significantly,  and  is  correlated  with  reaction  time  (RT).  The  distribution 
of  alpha  across  the  occipital  and  pairetal  areas  shows  remarkable  individual  differences 
in  both  symmetry  and  magnitudemris  is  not  caused  by  variations  in  skull  thicknesSt^e 
changes  in  this  distribution  during  the  course  of  performing  a  search  of  visual  memory 
suggests  that  the  effects  are  localized  to  visual  concx.  Performing  the  same  type  of 
analysis  of  activity  in  the  bet^ band  failed  to -eonfirnrthe  hypothesis  that  beta  activity 
replaces  alpha  activiQt.'Tfistead  beta  activity  shows  the  same  Idnd  of  reduction  in  power 
during  a  search  of  visual  memory.  However,  the  data  suggest  that  beta  activity  arises 
from  an  at  least  partly  different  neuronal  population  than  that  which  gives  rise  to  alpha. 

-::.,In  another  experiment  subjects  either  tried  to  form  an  image  of  an  object 
represented  by  a  visually  presented  word,  or  to  find  a  rhyming  word.  The  latter  task  did 
not  produce  a  change  in  activity  of  visual  cortex,  while  the  same  words  produced  a  pro¬ 
found  change  in  activity  of  visual  cortex, during  imaging. 

Acoustically  presented  words  were  also  found  to  produce  changes  in  MEG  activity 
arising  in  the  visual  areas  when  they  were  used  as  cues  to  forming  a  mental  image, 

Tonal  stimuli  were  used  in  a  Sternberg-like  paradigm  in  which  subjects  had  to 
search  memory  to  determine  if  a  recent  tone  was  or  was  not  am  member  of  a  set  of  previ¬ 
ously  heard  tones.  This  was  found  to  modulate  activity  arising  in  temporal  lobe,  and  not 
to  modulate  occipital  alpha  activity. 
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INTERIM  SCIENTIFIC  REPORT 

ATTENTION.  IMAGERY,  AND  MEMORY:  A  NEUROMAGNETIC  INVESTIGATION 

AFOSR  Contract  No.  F49620-88.K-0004 
by  Lloyd  Kaufman,  New  York  University 

INTRODUCTION 

This  interim  repon  describes  ongoing  research,  sponsored  by  AFOSR  under  Con¬ 
tract  No.  F49620-88-K-0004.  The  research  is  being  conducted  at  the  Neuromagnetism 
Laboratory  (NML)  of  New  York  University  and  at  the  Center  for  Neuromagnetism 
(CNM)  of  the  School  of  Medicine  of  New  York  University.  The  CNM  is  located  at  Bel¬ 
levue  Hospital  and  it  includes  two  7  channel  neuromagmetometers,  which  makes  it  pos¬ 
sible  to  meaure  the  neuromagnetic  field  at  14  positions  at  jne  time.  This  capability 
foreshadows  even  more  advanced  systems,  and  it  was  fully  utilized  in  two  of  the  major 
areas  of  research  described  in  this  report. 

This  report  outlines  the  progress  made  during  approximately  the  first  year  of  the 
contract,  and  it  forecasts  the  work  that  will  be  complet^  in  the  second  year. 

Evaluation  of  14-channel  Neuromagnetometry  System 

The  purpose  of  this  work  was  to  assess  the  precision  and  reliability  with  which 
dipolar  sources  are  located  using  a  14-channel  system  composed  of  two  7-channel  neu¬ 
romagnetometers  and  the  probe  position  indicator  (PPI)  used  to  locate  the  head  with 
respect  to  each  of  the  sensors.  The  attached  paper  provides  all  of  the  details  of  this  work, 
which  involved  using  a  spherical  "phantom",  a  plastic  model  of  a  skull  and  actual  human 
subjects.  The  precision  with  which  a  dipolar  source  could  be  located  within  a  spherical 
phantom  and  within  a  model  skull  was  found  to  be  less  than  3  mm  in  three  dimensions. 

Also,  the  determination  of  location  was  highly  reliable  since  20-30  independent  determi¬ 
nations  were  in  full  agreement  with  each  other.  The  human  work  demostrated  that  the 
equivalent  current  dipole  sources  of  the  magnetic  NlOO  component  of  the  auditory 
response  was  located  on  the  floor  of  the  sylvian  fissure  of  each  subject,  with  variations  in 
source  location  related  to  anatomical  differences  among  subjects,  as  revealed  by  MRI 
scans,  furthermore,  this  work  confirmed  earlier  reports  of  the  tonotopic  sequence  along 
the  floor  of  the  fissure.  The  reader  is  referred  to  the  attached  paper  (Yamamoto,  et  al., 

1988)  for  details.  The  essence  of  these  results  have  already  been  confirmed  by  Papin- 
icolaou,  et  al.  (1989). 

COGNITION  AND  BRAIN  ACTIVITY 

Background 

In  our  original  proposal,  which  was  submitted  to  AFOSR  on  30  July  1987,  we  out¬ 
lined  a  novel  approach  to  the  study  of  cortical  activity  associated  with  mental  imagery.  In 
the  previous  report  we  described  some  of  the  experiments  that  were  then  in  progress. 

Since  that  time  the  experiments  were  completed  and  the  results  of  some  are  to  be 
presented  at  the  forthcoming  Intemantional  Conference  on  Biomagnetism  in  August, 

1989.  The  abstracts  of  these  papers  are  attached.  Also,  one  paper  was  completed  and 


submitted  to  Science  for  review.  A  copy  of  that  paper  is  also  attached  to  this  report 
(Kaufman,  et  al.,  1989). 

In  all  the  experiments  described  in  the  attached  abstracts  and  paper  we  found  that 
the  magnetic  fields  associated  with  the  spontaneous  magnetoenchephalogram  (MEG) 
varied  profoundly  with  different  types  of  cognitive  tasks.  In  a  recent  article  on  computa¬ 
tional  neuroscience,  Sejnowski,  Koch  and  Churchland  (1988)  make  the  point  that  "it  is 
difficult  to  imagine  a  major  advance  in  our  understanding  of  brain  function  without  a 
concomitant  development  in  direct,  efficient  techniques  for  probinig  the  mechanisms  of 
distributed  processing,  (p.  1300)"  In  this  connection  they  mention  (MEG)  as  one  of 
several  new  methods  that  may  contribute  to  our  knowledge  of  the  computational  net¬ 
works  of  the  brain  that  make  complex  cognitive  processes  possible.  In  this  report  we 
describe  results  that  can  lead  to  the  realization  of  that  goal. 

The  cognitive  tasks  involve  the  manipulation  of  mental  images  based  upon  stimula¬ 
tion  by  physical  images,  visually  presented  text,  tone  bursts,  and  acoustically  presented 
words.  We  have  also  conducted  pilot  studies  on  mental  rotation,  and  plan  work  on  effects 
of  different  levels  of  processing  on  brain  activity. 

Modulation  of  Brain  Activity  During  Mental  Imagery 

To  summarize  our  findings,  as  described  in  the  attached  preprint  by  Kaufman  et  al. 
(1989)  we  measured  the  average  power  of  both  alpha  and  beta  activity  over  the  occipital 
and  parietal  areas  and  discovered  spatially  selective  suppression  of  activity  in  both  bands 
when  abstract  figures  were  briefly  presented  visually.  Subjects  first  saw  3  of  the  forms  in 
sequence.  Then,  several  seconds  later,  one  form  was  presented  for  100  msec,  and  subjects 
responded  in  one  of  two  ways.  First,  they  simply  pressed  a  button  whenever  they  saw  this 
final  "probe"  form,  regardless  if  they  had  seen  it  before.  Alternatively,  they  pressed  one 
button  if  the  form  had  been  one  of  the  previously  seen  set  of  forms,  and  another  button  if 
it  was  new,  i.e.,  not  a  member  of  the  memory  set.  The  first  condition  is  the  simple  RT 
task,  while  the  latter  is  the  choice  RT  task.  The  RT  in  tJ  .  latter  condition  was  much 
longer  than  that  of  the  simple,  and  the  duration  of  suppressi> in  both  MEG  bands  was 
commensurately  longer.  The  visual  evoked  field  did  not  show  the  same  effect  on  its  dura¬ 
tion.  Thus,  searching  visual  memory  results  in  long  periods  of  suppression  of  both  alpha 
and  beta  activity.  Furthermore  the  distribution  of  the  magnetic  field  associated  with  this 
suppression  effect  on  the  scalp  is  consistent  with  the  notion  that  activity  of  visual  cortex 
is  modulated  during  the  comparison  of  mental  images. 

There  were  some  ancillary  findings  thtat  could  not  be  described  in  the  previous 
Interim  Scientific  Report.  First,  the  distributions  of  alpha  band  and  beta  band  activity 
across  the  s^'alp  is  often  asymmetrical  in  a  given  individual,  and  differs  from  individual 
to  individual.  Furthermore,  the  variance  in  the  distribution  of  beta  band  activity  across 
the  scalp  cannot  be  accounted  for  by  the  variance  in  the  distribution  of  alpha  band 
activity.  This  despite  the  fact  that  both  bands  exhibit  a  correltated  suppression  in  ampli¬ 
tude.  Several  inferences  may  be  drawn  from  these  ancillary  findings.  First,  the  asym¬ 
metry  cannot  be  due  to  variations  in  skull  thickness,  as  is  stated  in  the  EEG  literature. 
Secondly,  beta  does  not  replace  apha  during  blockage  but  is  correlated  with  it.  Thirdly, 
the  sources  of  beta  are  in  large  measure  different  from  the  neural  sources  of  alpha.  A 
final  conclusion  is  that  the  suppression  effect,  if  it  is  related  to  classic  alpha  blockage,  is 
not  caused  by  elevated  arousal  or  visual  attention  pe  se,  but  is  in  large  part  is  related  to 


stages  of  processing  of  vision-related  mental  events. 

It  should  be  noted  that  the  literature  describing  the  background  to  the  foregoing  pro¬ 
ject  was  surveyed  in  the  preceding  Interim  Scientific  Report,  and  much  of  it  is  covered  in 
the  attached  preprints  and  abstracts.  Hence,  there  isno  need  to  repeat  it  here.  However,  it 
may  be  of  some  use  to  provide  a  brief  summary  of  the  methodology  employed,  as  it  is 
also  used  in  our  other  experiments  described  below. 

Comments  on  Methodology 

Kaufman,  et  al.  (1967;  1970)  employed  a  comb  filter  to  remove  all  harmonics  of  the 
fundamental  frequency  of  a  periodic  stimulus  fi"om  the  EEG  before  rectifying  and 
averaging  the  EEG.  This  method  involved  using  a  delay  line  and  subtraction  procedure  as 
well  as  a  complicated  procedure  for  producing  a  stimulus  (Kaufman  and  Price,  1967). 
The  analog  procedures  used  to  accomplish  this  involved  subtracting  the  activity  that  is 
time  locked  to  the  stimulus  from  the  ongoing  activity,  then  squaring  the  remaining 
activity  before  averaging  (an  analog  square-law  detector  was  employed).  This  gives  the 
power  (mean  square  voltage)  of  the  background  activity  and  shows  how  it  varies  as  a 
function  of  time  between  stimulus  events.  A  non-varying  measure  of  power  in  this  inter¬ 
val  implies  the  presence  of  "noise"  that  is  independent  of  the  evoked  response.  However, 
systematic  and  repeatable  fluctuations  in  power  as  a  function  of  time  between  stimuli 
implies  a  modulation  of  the  background  activity  by  the  stimulus.  The  powerful  digital 
computers  that  are  now  available  make  this  process  much  easier  to  implement. 

The  variance  about  the  mean  response  to  some  stimulus  is  proportional  to  the  power 
of  the  background  activity.  In  the  EEG  this  measure  (variance)  represents  mean  square 
voltage,  while  in  neuromagnetometry  it  represents  mean  square  field.  Consider  the 
definition  of  variance.  It  is  the  normalized  sum  of  the  squares  of  the  differences  between 
the  mean  response  and  the  field  (or  voltage  in  the  case  of  EEG)  at  any  instant  of  time 
within  the  epoch.  In  effect,  this  means  that  the  average  response  is  subtracted  out,  and 
the  difference  between  the  average  and  the  instantaneous  value  of  the  field  is  squared. 
This  value  is  added  to  that  of  all  of  the  other  values  occurring  at  the  same  time  after  each 
stimulus  event.  For  example,  if  the  brain  activity  subsequent  to  a  stimulus  is  sampled  256 
times,  there  will  be  256  variances  associated  with  each  average  response.  If  the  noise  is 
uniform  and  independent  of  time  after  a  stimulus,  then  these  variances  will  statistically 
equal  to  each  other.  However,  if  the  output  of  the  brain  should  vary,  depending  upon  its 
state  at  different  times  within  the  averaging  epoch,  the  variance  should  change  reliably  as 
a  function  of  time. 

With  this  information  in  mind,  it  is  easy  to  understand  our  basic  procedure.  A  sub¬ 
ject  is  presented  with  some  stimulus  or  some  set  of  stimuli  and  required  to  p>erform  a  well 
defined  task  with  respect  to  these  stimuli.  The  total  MEG  activity  between  0. 1  and  50  Hz 
is  recorded  throughout  the  experiment.  To  obtain  an  average  response,  a  particular 
stimulus  event  is  chosen  as  a  reference  signal  and  averaging  of  the  activity  time-locked 
to  the  stimulus  (both  before  and  after  its  presentation)  is  done  in  the  usual  manner.  Since 
vvc  store  all  of  the  ongoing  activity,  it  is  possible  to  compute  the  average  response  in  a 
narrow  bandwidth,  e.g.,  8-12  Hz,  of  the  digitally  filtered  MEG.  Because  of  the  narrow 
bandwidth  the  average  response  does  not  resemble  the  usual  evoked  response  but  we 
have  found  it  to  contribute  as  much  as  25%  of  the  mean  square  field  when  it  isnot  sub¬ 
tracted  out.  This  is  the  main  reason  for  not  following  the  procedures  devised  by 


Pfurtscheller  (1977).  In  an  event,  after  filtering  the  entire  record,  the  average  response 
and  its  variance  are  computed.  This  variance  is  the  critical  measure  because,  as  we  have 
stated,  it  shows  how  the  mean  square  field  associated  with  a  cortical  source  varies  as  a 
function  of  time  before  and  after  stimulation.  We  are  particularly  concerned  with  how 
the  fluctuations  in  variance  (field  power)  are  related  to  the  nature  of  the  mental  task.  Now 
let  us  turn  to  some  other  results  obtained  using  these  methods. 

Visualizing  and  Rhyming 

In  these  experiments,  conducted  by  Kaufman,  Glanzer,  Cycowitz,  Williamso  and 
Schwartz  (1989)  words  were  presented  near  central  fixation  and  subjects  were  asked 
either  to  form  an  image  of  th  object  represented  by  the  word,  or  to  find  and  silently  recite 
a  word  that  rhymes  with  the  word  that  had  been  seen.  The  motivation  was  provided  by 
the  inference  of  Shepard  and  Metzler  (1971)  that  visual  cortex  is  involved  in  manipulat¬ 
ing  mental  images.  Again,  we  used  modulation  of  alpha  as  an  index  of  whether  or  not 
visual  cortex  was  involved  in  the  two  kinds  of  mental  operations  -  one  involving  images 
and  the  other  a  language  related  task.  The  details  are  given  in  the  attached  abstract. 

We  have  already  gone  beyond  the  findings  described  in  the  abstract.  We  now  have 
data  supporting  the  hypothesis  that  alpha  is  suppressed  by  presenting  easily  imaged 
words  even  though  the  subject  is  performing  the  rhyming  task.  The  depth  and  duration  of 
the  suppression  is  intermediate  between  that  which  occurs  when  the  subject  is  activively 
visualizing  such  words  (the  maximum  effect),  and  when  the  subject  is  merely  looking  at 
a  nonsense  word  (shortest  duration  effect).  We  intepret  this  to  mean  that  there  is  an 
immediate  perceptual  processing  that  is  modality-bound,  but  once  the  processing 
involves  a  mental  image  the  visual  cortex  continues  to  "work".  However,  this  "work"  is 
terminated  rapidly  when  the  task  is  translated  to  the  linguistic  domain,  so  that  the  effect 
in  the  rhyming  task  resembles  that  associated  with  nonsense  words.  This  suggests  that 
effects  should  be  found  in  temporal  lobe  when  subjects  are  performing  the  rhyming  task. 
We  intend  to  follow-up  on  this  conjecture. 

Modulation  of  Activity  of  Auditory  Cortex 

The  preceding  experiment  caused  us  to  consider  the  possibility  that  activity  of  other 
brain  areas  may  be  similarly  modulated  by  the  performance  of  a  cognitive  task.  Toward 
this  end  we  designed  an  experiment  in  which  three  different  musical  notes  were  played  in 
sequence,  and  another  "probe"  note  was  played  2  seconds  later.  As  in  the  first  experiment 
described  in  this  report,  subjects  had  to  determine  if  the  probe  note  had  or  had  not  been  a 
member  of  the  original  memory  set.  There  was  one  major  difference  in  procedure,  how¬ 
ever.  Subjects  were  shown  a  number,  1,  2,  or  3  about  2  sec  before  each  memory  set.  The 
number’ 1"  meant  that  they  had  to  store  only  the  last  of  the  three  notes  for  subsequent 
comparison,  ’2’  that  they  had  to  remember  the  last  two  notes,  and  ’3’  that  they  had  to 
remember  all  three.  Thus,  we  were  able  to  manipulate  the  size  of  the  memory  set  while 
making  no  change  at  all  in  the  physical  stimuli.  The  different  size  memory  sets  were 
mixed  at  random,  and  responses  depooled  later  so  that  w  could  examine  the  effect  of 
memory  set  size. 

To  be  brief,  we  found  that  activity  in  the  8-12  Hz  band  originating  in  temporal 
areas,  probably  auditoiy  cortex,  was  indeed  modulated  during  the  search  of  acoustic 
memory.  Control  measures  failed  to  reveal  any  corresponding  change  in  alpha 


originating  in  visual  or  parietal  areas.  We  also  have  preliminary  evidence  that  the  dura¬ 
tion  of  the  suppression  is  larger  for  longer  set  sizes.  This  work  is  continuing. 

Visual  Memory  Set  Size 

We  are  concurrently  conducting  an  experiment  similar  to  that  above,  but  using  the 
visual  polygons  of  the  first  experiment.  Instead  of  musical  notes,  we  present  three  of  the 
polygons  in  series  shortly  after  either  a  high,  medium  or  low  pitch  tone.  The  low  pitch 
tone  means  the  subject  has  to  remember  only  the  last  of  the  polygons,  and  so  on.  This  too 
permits  control  over  memory  set  size  without  affecting  the  physical  stimuli.  In  most  sub¬ 
jects  there  is  a  monotonic  increase  in  duration  of  suppresioii  with  memory  set  size.  We 
have  not  yet  correlated  this  with  RT,  nor  have  we  yet  separated  correct  from  incorrect 
responses.  This  will  be  done  shortly.  However,  we  do  know  that  "new"  stimuli  (those  not 
in  the  memory  set)  give  the  same  duration  of  suppression  as  do  "old"  stimuli,  thus 
confirming  Sternberg’s  result  that  the  search  of  visual  memory  is  exhaustive. 

Auditory  Word  Experiment 

In  the  last  report  we  described  an  effort  to  produce  words  using  a  speech  syn¬ 
thesizer.  We  were  finally  successful  and  have  just  begun  to  collect  data  using  auditory 
words  that  are  either  imageable  or  abstract,  which  allows  us  to  do  an  experiment  that 
complements  the  one  described  in  the  abstract.  Preliminary  trials  show  that  imaging  the 
object  represented  by  an  imageable  acoustically  presented  words  results  in  specific 
supression  of  visual  activity  in  the  alpha  band. 

Forefcast 

The  major  forecast  is  that  we  shall  complete  all  of  the  experiments  described  above 
that  are  as  yet  incomplete  and  prepare  papers  for  publication.  In  addition  we  plan  a  new 
experiment. 

In  this  new  experiment  we  shall  present  letter  stimuli  as  in  the  classic  Sterberg  para¬ 
digm.  We  shall  use  1  to  6  different  letters  and  a  target  letter  that  could  be  either  new  or 
old.  There  will  be  one  differene,  however.  Sometimes  the  target  letters  will  also  be 
presented  in  a  different  case,  i.e.,  lower  case  if  the  memory  set  letters  were  capitals,  and 
vice  versa.  This  will  allow  us  to  test  the  hypothesis  that  when  a  visual  match  is  not  possi¬ 
ble,  there  will  be  concomitant  activity  in  the  auditory  or  speech  areas,  and  such  activity 
will  be  less  likely  when  a  visual  match  is  possible. 


